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ABSTRACT 



Considerations and procedures £or applying Performance^-Struc ture Or^.^ired 
CAI in Naval training are described^ in terms of a general diagram o£ the 
nticessary elements in a CAI system* 

The fundamental objective is to lead the student to develop his own 
cognitive structures that vill serve him £or generating the surface 
structures of tasks • The instructional system must accomplish this by 
generating and scheduling a suitable instructional sequence for each 
student . 

The developmental steps, from job task*structure analyses to computoi.* 
programs, are described. Techniques for generalizing these procedures 
£all into two categories: higher-level programming languages that would 
permit more rapid production o£ programs for specific applications, and 
general-purpose programfi that accept data modules specific to an appli- 
cation. Both kinds of techniques have been used in earlier applicationf; . 

Examples of instructional flowcharts, programmer's flowcharts, and 
data encoding procedures used in the development o£ the RIO trainer are 
presented to illustrate key points in the developmental steps. 
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FOREVTORD 



'I'he University of tiouthern California and the NaVal Training Equipment 
Ct:^ntei^t under ARPA fundings are cooperating in an effort to develop a system 
which will serve to facilitate the production of appropriate Computer-Aided 
Instruction (CAI) for the Navy. This system^ basically taking the form of 
a model or theory of GAI^ is in part derived from and evaluated by empirical 
studies as described in other Naval Training Equipment Center technical re- 
ports by the present authors (Rigney^ et. al.j 19731 Rigney^ et. al.i in 
preparation). Because this empirical aspect of the research exhausts major 
portions of the resources available for the project^ explication of the 
system contained herein is in its earliest stages. It is published here 
mainly for heuristic reasons and is not intended to be used as a refined 
set of guidelines for developing CAI materials. 

Even in this initial form^ however^ the system can suggest to course 
authors various components of CAI and gross steps associated with the devel- 
opment of these components that need to be considered in the process of 
course* construction. Further development of the system will consist of en- 
deavors to expand upon current capabilities by offering: (a) specific in- 
structional approaches for the components forming the structure of CAI; (b) 
computer programs^ capable of implementing the suggested instructional ap- 
proaches^ which are general enough to apply in a range of subject matter 
areasj and (c) computer capabilities for generating some portions of the CAI 
specifications for a given application. Thusi CAI program developers will 
be able to use the system as an aid to deciding upon instructional approaches 
appropriate for their particular teaching objectives. Further, they even 
will be able to obtain computer programs which essentially are ready for ap- 
plication in their training program. 

The development of generalizable instructional appi^oaches and the sup- 
porting computer programs is a prime reason for viewing the extensive CAI 
course construction activities as essential to the project. The empirical 
research^ however, serves additional important functions^ among which is the 
ancillary contribution of developing cost-^effective .means for teaching 
skills for critical Navy jobs. The Radar Intercept Officer*s job was the 
first technical area addressed in this way. Developing CAI materials for 
teaching the utilization of the AWG-9 system for maintaining the F-lU air- 
craft being considered for the continuation of the empirical aspect of 
the research on this project. 




ARTKUB S. BIiAIWES 
Scientific Officer 
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Variations Ln entering skills and knov^ledge are currently handled by 
trials-to-criterion logic. IT a student can perform the oasle.it oL" a 
graded series ot" problems, he is transitioned to the next level of problem^s. 
Under certain circumstances, more powerful auaptlve control techniques 
inay bo applicable; e^g*, some variation of dynamic programming, a mathema- 
tical technique for optimization. Variations in learning rates and in 
patterns of essential subskills are currently handled by vertical itera- 
tion through hierarchically structured material, combined with trlals-to- 
crlterion logic. Within a block of problems, a student may, for example, 
be required to repeat a problem under conditions which unburden him of 
performing all but one of the subtasks required in the problem, so that he 
can concentrate on doing that one correctly* 

This concept of how to train personnel to perform has been implemented 
In an individual skills trainer for the Radar Intercept Observer (Rigney, 
et alt, 1973b) and in a program for giving students practice in trouble- 
shooting electronic devices (Rigney, et al*, 1972a). 

The problem for the developmental model to be described here is to 
provide ways to specify the essential elements for applying PSO CAI in a 
particular context. To do this, it is necessary to know in great detail 
the cliaracteristics of the performance that is to be the subject of the 
training. What is it that the student must learn to do? 

When this has been described, the information can be used for generating 
specifications for a CAI system. In this guide, the diagram of the essen- 
tial elements in a CAI system given in Figure 2 will be used to identify 
the "bases that must be touched" in the generation of these specif ication^.^ . 
This diagram has been the subject of recent work on a "CAI testbed" concept 
(Rigney, 1973c). 
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SECTION II 



CRNERAL CAI SYSTEM CONSIDERATIONS 



In this section^ we \>tlll develop the considGrationi; for determining 
the type of CAI system, in terms of particular examples of the elements, 
as outlined in Figure 2^ that would be needed for applying PSO CAI in a 
selected context. In most cases, the different configurations could be 
produced by software^ although there also could be requirements for 
special types of peripheral hardware in addition to standard terminals^ to 
be interfaced with the basic processor* We will attempt here to lay out 
the considerations for each element in the system^ so that procedures for 
generating specifications can be formulated in the next sections* 



Internal Processing 

This box represents the student* Of course^ just as we do for other 
elements in the CAI system^ we must develop "specifications" for the 
student. Implicit assumptions about student characteristics must be made 
explicit, Therv always is some definable population of students^-some 
"target" population^ for whom the CAl system is to be designed. We need 
to know characteristics of this population in relation to prerequisites for 
the course^ and in terms of learning abilities. If we are going to adapt 
to Individual differences among students with respect to entering skills 
and knowledge and with respect to rates of learning, we need to know some- 
thing about these student characteristics. 

As a form of drill and practice^ PSO CAI assumes the students already 
are Eamiliar with the "theory of operation" or are learning it in a parallel 
course. This form of CAI normally would be embedded in the total curriculum. 
Therefore ^ tfie scre^^ning that is done in the broader context should suffice 
for prerequisite requirements. Rate of Learning might be predicted from a 
mixture of achievement and intelligence tests. However^ it is preferable 
to use adaptive control mechanisms to sense important individual differences 
and to adjuLt to them as the student is progressing, rather than to depend 
upon the classical psychometric approach^ viz^ batteries of tests and multi- 
ple regression^ to try to predict what "treatment" should be assigned to 
each student. 



Student-Program Interface 

This Includes stimulus display^ responses^ and response records. 
Observe that responses include "trial responses" the student may make covertly 
before lie commits himself to an observable response. 

The problems here are to identify the stimulus displays^ the response 
structures^ and the response records that will be required. There will be 
many questions to be answered^ and it is probable that work on this part 
of the specifications should be deferred until other parts of the system 

erJc -4- 
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havi* bcvn analyzid* for c^^amplo, wo woitld not know wliat locord aboiiL 
rc:tj|>onst s until kiKiw what wc want to tlo with thi^>sij data, wlilch JciuMttls 
on tho tyjic' of adaptive contrnJler usod, ami the typt^ of statistical 
ana lys is Lliat iri rc:([u i red . 



Student Data 

ThLs includes the sCudent HufficicnL history computer and studenc 
records, Thi* latter would include processed response records plus other 
information abouC Che student; e,g,, intelligence tesc scores, 

A word or two about the concept of a student sufiicieut historV com- 
puter is in order. The concept oE a sufL'icient history was discussed by 
Atkinson and Paulson (1972), They stated that, '*An index summarizing', tite 
information in a student *s- responi^e protocol is a sufficient history iC 
any additional information from the protocol would be redundant in tlie 
determination of ttie student's state of learning. The eoncept is analagous 
to a sufficient statistic,,," Ue add the word "'computer'^ to indicate not 
a piece of hardware, but software required to compute the Sufficient histoiy 
from response records and student records. For example, a Markov decision 
model we are developing (Wollmer, 1973) for adaptive control over the 
instructional sequence requires, for a student's sufficient history, records 
of the number ot trlali? the student took at each level o£ a hierarchically 
organized course, the number of successful trialt^ at each level, and tlie time 
spent in a level. From these data, collected from a preliminary sample 
ot students, the model allows the computation of optimal patterns; of success- 
ful trials Lo require of the next student in order to mlnitiiize time to 
complete the course. Each student's sufficient history becomes part of 
the infornLitioa usi'd in the optimization for the next student* 

The same rciiuirks about priorities of analysis apply here: What consti- 
tutes a r;Ludent sufficient history will be determined by what is to be done 
with t[»es<* data, which will depend on who will use the data and what they 
will be ii-^oti for. Several different sufficient histories may be needed. 

Adaptive Control Icr 

Thus Includes content/mediator files, essentially storage of course 
£na tor la Is , an instruc tiona 1 sequence opt imizer, an ins true t ional sequence 
sche<lnlcr, and an Instructional content generator. These three instructional 
operators require some explanation. An optimizer would be some method for 
ImprovinK the effectiveness of the CAl system, by optimizing learning rates 
under s^ome set of constraints. Atkinson and Paulson (1972) have described 
gener;il procedures for going about doing this. 

The optimizer would identify an optimal instructional sequence for each 
individual student. The composition and sequencing of instruction is the 
principal way, if not the tinly way, to influence learning ond retention in 
tfie CAl system. The optimizer would control an instruction;il sequence 
schetluler. If no optimizer is used, there still must be a scheduler, 
instruction must be f^equenced by some mechanism and that meehanism must be 
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(Ic^^trib^ni III tUo £;pec i£ icat Ions for the CAI system, 

Lt is not yet clear, at this point in the technology, how general an 
opilmi^er-schodult^r can be nuide, Atkinson (Atkinson and Paulson, 1972) has 
combin<.^d several diCforcnt tectiniques for adaptive control over the sequenc- 
ing of the ^;trand!i in his reading program* it seems likely that this will 
be tiie case for other programs* General mathematical techniques be 
represented tn algorithms that must deal with highly specific data, learn- 
ing (nodel«, and^ possibly, constraints. 

Tho instructional content generator could be either of two types: 
software that "makes up" the instruction on the spot from simple descrip- 
tivo data, as iu the TASKTEACH programs, or a team o£ Instructors who 
write programmed instruction frames, and generate the entire sequence ahead 
of time. 

Unfortunately, although writing CAI frames is a most time-consuming and 
<»xp( nsive process, the range of applications in which the instructional 
content can be dynamically generated by a computer program currently is 
quite narrow- In the drill and practice mode it is easy to generate a 
series of math problems by chariging the values of variables in a few stand- 
ard problem structures. It is considerably more difficult, but still qaite 
possible as we have shown, to generate the instructional sequence for a 
troubleshooting problem step-by-step, while the problem is being solved by 
a student, and to do this for troubleshooting a wide variety of devices. 
The only restriction on the range of devices is that certain of their charac- 
teristics and certain relationships In them be representable in a standard 
data-structure. It is possible as we al£>o have demonstrated, (Rigney, et al., 
1972a), to represent the structure of tasks in simple data-structures and to 
provide a general program to teach students how to perform a variety of 
tasks, either allowing them to compose their own individual instructional 
sequences, or placing the instructional sequence under program control. But 
It is not yet possible, so far as we know, to create a program to d£ 
task analysts or to generate the surface structure of specific tasks from 
a general, or primitive data structure analogous to Chomsky's (1964) deep 
structure. To do this, we would need a detailed model of how humans 
process information and organize their performance. 

For the instructional content generator, the two major questions are: 
"What is the surface structure of the performance to be taught?" aad ''What 
are the content mediators that will be required in the training?" Content 
mediators are the operations and concepts in the material to be taught 
that bridge between stimulus and response. For example, in algebra, the 
mathematical operations and mathematical concepts required to solve a 
quadratic equation would be content mediators. We use this term to distin- 
guish between these and learning and memory mediators, which are general 
operations the student may perform on broad categories of material to improve 
rate of learning and length of retention. 

The content/mediator files in the adaptive controller store the material 
that is to be organized Into an instructional sequence, or that is already 
organi?.ed into some "lesson" format. Thft way this material is stored and 
liow iiiuch oJ it there is to be stored are important considerations. CAI 
requires a lot of storage, which usually takes the form of disc files. The 
random-access disc file is a common and almost indispensable feature of 
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timo-iihariiig Hystt.*ms. Howevor, some CAl material may be stored more econom- 
ically in peripheral devices in the t'orm o£ slides of microfiches. 

The npti[ni?.(?r, schedultir, aad generator mecharastus in the adaptive 
controller tend today to be relatively crude procedures. These are anas 
that should receive much more intensive research and development, since 
tliey are at the core of the CAI system, and will determine its effective- 
ness . 



Instructional Sequence 

This sequence contains knowledge of results, subject-niatter, and 
external mediators* It is the "input tape'* to the student, We distinguish 
l>etween knowledge of results a^ something the system provides for the stu" 
dent, and internal feedback as something the CNS provides, using information 
that comes to It both from the external and the internal environments* 
Knowledge o£ reiSults provides some information for internal feedback, but 
thi.s information may or may not be used* The knowledge of results provided 
by the system sometimes may be superfluous. 

The two types of external mediators, content mediators and learning and 
memory mediators, have been defined earlier* Since most material is at some 
intermediate difficulty level in a roughly hierarchical structure, there 
usually are a number of different concepts, relation^^, and skills to be 
learned,. These must somehow be organized into a sequence and presented to 
the student's very limited input system in serial order. Thus, the funda- 
mental problem is how to schedule the instructional sequence in a way that 
will cover tlie different conc(?pts, rules, and operations to be taught, and 
that will lead to optimum learning and retention rates. 

Ay represented in the diagram, the instructional content would have been 
already composed by tho generator and would be put into serial order by the 
scheduler^ both in tho adaptive controller. Thus, the instructional sequence 
Is the output of operators In the adaptive controller. 



Feedback Ta)Ops 

The CAl ^systen* contains information channels that could be used to learn 
abnuL the studotit and to learn about the adequacy of the instructional se- 
queace, as well arf to instruct and to provide knowledge of results to the 
student. Devising ways for the CAI system to learn about the student has 
turned out to he aa extremely difficult thing to do. The Stanford CAI 
group (Atkinson and Paulson^ 1972) was the first to tackle this and have 
done the most definitive work on this problem. 

These luetn>ack loops may be essentially open, because of long time delays 
or essentially closed. For example, the feedback loop providing information 
about the ndequ<icy of the Instructional sequence usually ia open. The very 
lon^ time required to revise instructional materials usually means the 
revised version will be used on a different sample of students. For an 
cXLeiislvu and provocative treatment of the topic of feedback in human behavior 
see Powers (1973) » 
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SECTION III 



PROCKDIIKI-S FOK GENEItA'LING SPECIFICATIONS FOR A PSO CAI .SYSTEM 



It is cloar that the determination of what is to be taught requires 
identification and analysis of the tasks that the student is to learn to 
perform to som« criteria of proficiency* This analysis will be the prin- 
cipal source of specifications. However, external objectives for the system 
also mu«t be* considered. Whoever is the customer for the system may want 
certain types of data from it which must be provided for in the specifica- 
tions. For example, the customer is likely to want records of nucnber of 
students put through the course, student scores, costs per student hour, 
etc. If the system is to be used for research objectives, or if there is 
to be an initial evaluation of it, these objectives are likely to have 
special requirements for data that usually exceed those for student 
sufficient histories for the customer. 

These are, however, secondary considerations in comparison to the 
determining effects on the nature of the CAI system of the requirement to 
simulate essential aspects of the job performance-environment. Therefore, 
we shall tackle the task analysis problem first. We shall use three terms, 
surface structure, content mediators, and learning and memory (L & M) media- 
tors, to divide the task analysis problem into three categories. We prefer 
these terms to training objectives and enabling objectives. Roughly speak" 
ing, the surface structure of a task performance will be the (mostly) observ- 
able actions of the performer, performing in a specific situation on a speci- 
fic interface. Thus, the action on the front-panel controls and indicators 
required to tune the AN/URC-32 transmitter would be classified as surface 
structure. What the student has to know in order to tune this and other 
similar transmitters would be classified as content me^Hators. When he has 
learned enough of these^, he can use them to generate the surface structure. 
How the student goes about learning and remembering content mediators are 
the functions of L & M mediators which the student happens to know how to 
use already or which are incorporated in the instructional sequence. It is 
possible, we believe, to teach learning and memory (Strategies to students. 

As a consequence of learning, the student develops his own deep struc- 
ture to guide him in performing without instructional assistance. The 
structure of this personal deep structure is unknown, of course. Neverthe- 
less, we can, by using Instructional techniques, assist students in develop- 
ing deep structures. Our problem here is to develop procedures for identify- 
ing the implications of surface structures^, content mediators, and L & M 
mediators for the six groups of elements in a CAI system, as diagrammed in 
Figure 2. This requires that we become concerned with the categorization 
of features as well as with specific details of these features, 

PSO CAI is concerned with giving personnel drill and practice in per- 
forming technical jobs in which the boundaries of the performance are well" 
defined and performance cycles through well-defined episodes. This is 
characteristic of most operator and maiatainer jobs. These jobs also may 
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bo ombodded In an organi'^ation or system vhich requires tUe coordinated 
perCormances oC many individuals and which controls the time and place 
oC performance, Tlie episodic^ or cyclical nature o£ critical performance 
requiromentti can be diagraminod as in Figure 3, 

PREPARATION 

* 

OPERATIONS 
RETURN TO STANDBY 

' 1 I 

Figure 3, Overall Performance Cycle 



Our principal concern in ?S0 CAI Is with the operations part oC this 
cycle, although in pre-CAI days we thoroughly analyzed the entire cycle 
for some CIC (Bryan, et al», 1956a, b) and maintenance (Rigney and Bond> 1964) 
jobs. The operations part o£ this cycle can be analyzed (as could the other 
two parts as well) into more and more detail. In terms at goal-action 
hlerarchie<3, as in Figure 4* 




TOP GOAL 
SUBGOAL N 



ACTION SET N 
SUBGOAL N-1 



ACTION SET N-1 

T- 

SUBGOAL 1 



ACTION SET I 
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Figure 4^. Performance as a Goal-Action Hierarchy 

-10- 



KAVTRAEQUIPCEN 73-C-0065-1 



The strui Lure of the perforiivince can be analyzed in this way down to the 
industrial engineer's titno and motion units (Rigney and Townc^ 1969) if 
tliat is deii ired , 



Cate^or i y.nl ion of Surface IF^tructure Features 

[t is clear that surface structure features are major detertninants oC 
hardware design^ particularly of the type of student terminal that would 
be required^ ^^ince the terminal is the primary substitute for the performer 
per formance-environment interface, 

l^or the purposes of generating specifications for a CAI application, 
surface structure features of tasks can be categorized on the basis of 
interface L'haracteristics with which the performer interacts. He receives 
information from various types of sources in the environment and he operates 
upon various type,s of objects in the environment. These may be characterized 
in a preliminary way, as outlined below, 

1, Types of Displays in the Performance Environment 

1 . 1 Man-Mac iiine Inter faces -^Specific Displays 
CRT Graphics 

CKT AXphanumerics 
Kront Panels 

1.2 Extended Memories 
Tex t 

Diagrams 

1.3 General Environment 
Visual 

Auditory 
Toueh 

1 .4 Static or Interactive 

2, Responses in the Performance Environment 

2 . 1 Locuti 

2.1.1 Man-Machine Interface 
discrete Controls 
Continuous Controls 
Tracking Controls 

2.1.2 Graphics Surface 
Writing 
Pointing 
Drawing 

2.1.3 General Environment 

2 . 2 Type 

2-2,1 Reaching/ Manipulative 
2,2,2 Locomotor 

3, Special Lexicon 

3.1 Verbal Symbols 

3.2 Graphic General 

3.3 Graphic MatUematic 
3 -4 Programming 
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Categorlgation oi' ConLent and L & M Mediators 

In the ideal ca.se, we would be able to describe *?uitablt: deep strtictiircs 
that students should possess • Consider, for example, the instructional 
time that is wasted in teaching students "theory^' in training courses that 
turns out not to be needed or even useful for their job performance because 
we do not know the true nature of deep structures. Gagne (1970) pioneered 
thinking about the nature of deep structures in education and training. 
There is a fascinating and growing literature on the subject. This litera- 
ture, up to about 1970, was reviewed in an earlier report (Rigney, 1971). 

Subsequently, there have been striking developments in modeling the 
structure of long-term memory; e.g., Runielhart, Lindsay, and Norman (1972), 
Anderson and Bower (1973), which are excellently summarized in Anderson 
and Bower (1973). This work is leading toward eventual simulation of 
human cognitive processes, which has enormously important implications for 
education and training. Eventually, we may be able to represent deep 
structures stored in long-term memory and to operate upon these to produce 
surface structures in ijays characteristic of human learning and memory. 
An example of a prepositional deep structure representation of a sentence, 
used in Anderson and Bower's model. Is given below (Figure 5). 




PARK PAfiT HIPPIE TOUCH DEBUTANTE 



Figure 5- An Example of the Prepositional Representation of the 
Sentence: "In a Park a Hippie Touched a Debutante". 
From Anderson and Bower (1973), page 139 



C - Context F - Fact P = Predicate 

L ^ Location S = Subject R = Relation 

T = Time 0 - Object 

£- Set membership; a, b, .... i = Nodes 

(Nodes represent ideas and the links represent relations or 
associations between these ideas) 



Language is a form of performance* Written or spoken sentences are 
said, since Cliomsky (1964) originated the concept, to be the surface structure 
of language. This is what is seen, heard, or spoken. But this surface 
structure can vary a great deal and still convey the same meaning. 
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Scntoncet; can be paraplirascd witKouL destroying the essi^ntial information 
thoy communicate. There i«, Chomsky said, a deep utrutturo that conveys 
the mi}aning, and that is tho basis for generating the i^urface structure 
of lac^UfigG * 

An artalogy cMn be drawn betwet^n other forms of human pt^rforniauce and 
language pcrformanCG, Beginners must istumble along or be led by tho hand 
until they learn a deep structure that will let them generate the desired 
performance, PSO CAI is designed to give them practice in generating this 
performance. It is dosigndd to be extended back into the structure of con- 
tent mediators by providing drills in sub;;kills which underly t\w stirface 
pt^rformance. It gives the student the opportunity to organize what he 
already knows and to identify what he needs to know to generate the surface 
structure of whatever performance is required for the job. We propose that 
there are powerful self-organizing processes In the central nervous system, 
that function in ways as yet very poorly known, to develop deep structures* 
These self-organising processes have the opportunity to function during prac- 
tice in performing surface structures* In drawing this analogy, we also 
should point out the obvious fact that language is used in learning to perform 
tasks. As a representational process, as a means for comnunicating instruc- 
tions, as the basis for creating shared contexts which allow humans to 
^'stand aside and look" at what they have done, are doing, or are going to do, 
language is a universal and indispensable learning tool. In fact, it is 
possible that tiie same deep structure in long-term memory, what Anderson 
and Bower (1973) call the "strategy-free component of memory" is the basis 
for all cognit ively-control led performance. 

For the present, however, our objective must bo the identification of 
important categories of content mediators embedded in the surface structures 
of the tasks to be taught, do that appropriate instructLonal sequences can 
be composed and scheduled, the appropriate subroutines can be put into the 
computer program to give drills in learning these content mediators; the 
appropriate stimulus; displays can be designed, etc. 

At this point in history^ the analysis of task content mediators is very 
much an empirical matter* Some rather broad assumptions must be made about 
the common cultural skills and knowledge possessed by the target population 
of students* Only the content mediators which become apparent from the 
surface structure analysis should be examined* These could be divided into 
two groups; representations and operations* Representations are models of 
and abj^cractionfj from environmental features; concepts^ objects, processesj 
aad events, that students must understand to be able to perform. Operations 
nrt: information-processing operations the student niust be able to do men- 
tally to be able to p<srform. They may be classified in many ways, as 
various '^taxonomies;*' Gagne (1965); Bloom, et al., (1956) testify* 

Taxonomies like these would be useful for generating specifications for 
software for CAI only to the extent that they suggest the directions that 
more detailed analyses must take, or identify groups of operations that 
have been or could be implemented in computer programs. Otherwise, they 
are entirely too general* 
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'Ihese classifications and taxonomies tend to overlook the structurL^s 
of operations. For example, there is a certain order of information- 
processing events character t ic of particular content mediators. Indeed^ 
knowing thti sequence of operations allows the student to predict **what to 
do next'* and to program himself to do it. The analysis of content mediators 
must become concerned with the sequences of operations and with ways to 
represent these to the student > for these serial patterns are themselves 
content mediators. We have done a considerable amount of work on this 
problem, A computer program has been produced to teach students the serial 
structure of operations on man-machine Interfaces (Rigney, et al,> 1972a) 
which Is a step in the direction of a "task parser" analogous to a sentence 
parser used in natural language processing. 

The program, called CAPTIVE > operates on a list of action and goal 
statements organized by a numerical code that describes the syntax of the 
performance In terms of action and goal sequence constraints^ goal set 
membership of actions, etc. The program can recognize vhen a student's per- 
formance is '^grammatical", that Is, when it follows the rules of syntax. 
It can take a string of actions apart somewhat like a sentence^parser^ but 
it goes far beyond a parser In its ability to guide a student In learning 
to use the correct syntax. 

Since the principal implications of content mediators for CAI system 
specifications are for computer programming^ analysis must become quite 
detailed. The principal analytical tool to be used is the instructional- 
sequence flowchart. Ideally^ this would make content mediators explicit 
along with the specification of surface structure considerations. 

Learning and memory mediators are more or less "content-free" operations 
that the student could perform to Improve his rate of learning and/or his 
length of retention. We say more or less content-free because it is unlikely 
that all L 6t M strategies useful for learning mathematics, for example^ 
would be equally useful for comprehending and remembering narrative struc- 
tures of history or literature. 

Recently^ there has been a revival of interest in the effects of imagery, 
and to a lesser extent in other learning strategies^ stimulated in part by 
demonstrations of the startling effectiveness of these strategies in learn- 
ing and remembering nonsense syllables (e,g,> Bower, 1972; Prytulak, 1971), 

It docs seem important that L 6 H mediators should be Incorporated in 
CAI systems, insofar as the experimental evidence suggests that they would 
be worthwhile. It is even more important that researcli on the effective- 
ness of these strategies be done in the context oi: CAI> since this is quite 
different from the usual laboratory setting. 

It is time we stop categorizing human students with simpler laboratory 
animals like rats and pLgeonS and recognize that animal conditioning 
paradigms are insufficient bases for CAI, The mental operations that stu- 
dents; per i'orm while learning are crucially important determinants of how 
well students learn and remember. Our problem is to acliieve a better under- 
standing of wliat these mental operations are, and to gain enough control 
over them to increase the effectiveness of CAI, 
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l\)L' tliLii reason, we be I Love L 6c H mediators should be inr.KKlijd Lii 
plannl.n^^ even though knowledgi^ about what they art? and how to uiie them 
outsidt^ of tfie experimental lahoratory ii^ still very sciint. We iiiclude a 
brief .sumntary of ttiL^ rt'searcii literature on Imagery^ an example of an h 
6c M mediator, in llio followLti^ paragraphs* 

The image-evoking (I) value or concreteness of stimulus material is 
positively correlated with the learnability of tliat material* Recall and 
recognition are greater for material of high I value, (Paivio and Madigan, 
1968, 1970; Begg and Paivio, 1969; Paivio, 1969; Paivio and Csapo, 1969; 
Paivio and Yuille, 1969; Paivio, YuiUe, and Rogers, 1969; Paivio aad Rowe, 
1970; Paivio and Smy the , 1971; Anderson, 1973; Craig, 1973; Griffith and 
Johnson, 1973; and Montague and Carter, 1973) » 

Experimental manipulations of imagery affect learning. Two types of 
variables have been studied, (1) Instructions (or prior set) to form 
interactive images increases recall and recognition o£ the to-be-learned 
material, (Paivio and Yuille, 1967, 1969; Bugelski, 1968; Bugelflki, Kidd, 
and Segmen, 1968; Bower, 1970; Anderson, 1971; Anderson and Hidde, 1971; 
Bower, 1972; Anderson and Bower, 1973; Griffith and Johnston, 1973; and 
Levin, 1973,) (2) Providing mediating pictures facilitates learning* 
(Davidson, 1964 ; Reese , 1965 ; Rohwer ^ Lynch, Levin, and Suzuki, 1967; 
Lippman and Shanahan, 1973; and Nelson and Brooks, 1973,) 

Individual differences in effective imagery potential affects learning. 
Individuals classified as high in ability to Image recalled or recognised 
more than those rated lov in imaging ability* (Ernst and Paivio, 1969, 
1971; Sheehan, 1966, 1971; Paivio and Okovita, 1971; and Marks, 1972») 



Implications of Task Analyses for CAI System Specifications 

The outlines of a hypothetical mechanism for generating CAI system 
specifications are illustrated in Figure 6* Each different feature in the 
task structures* might be "run through" the impact analyser to see if that 
feature would have an impact on any of the six CAI system elements, coasider- 
ing each in turn. If there would be no impact on an element, the next 
element would be cousidpred for that feature, and so on, until an element 
was found that was impacted by the feature. In that case, the hardware 
implicationp^ analyser would be called to analyze the implications of the 
feature for that cleinent. There would be sub-analysers for major hardware 
components; m*imory, CPfJ, and terminal sub-analysers are indicated in the 
figure. The c^utputs of the implications analyzers would go to the specifi- 
cations generators^ which, in the case of hardware, would look for matches 
between requirements and available items, and could list alternatives • For 
example, animated, interactive graphics might be desirable, if not required, 
to illuf;trate processes to the student, and to engage him in exploring these 
processes, say the solid-state physics that are the basis for transistors* 
Two types of graphics terminals might be Jisted in a '^catalog" of hardware 
items known to be on the market. This catalog would be in the form of a 
data base which the terminals specif ication generator would search* 
Suppose one of the terminals provided a lighr: pen with very accurate X,Y 
coordinate discrimination and the other provf^ded a touch panel with crude 
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X,Y coordtnatt^ (iiscrimina t ion . The terniinal spec i£lcat ion generator would 
need dat;i dL^acrlbLnj* the niaximiini accuracy of X,Y discrimination ever re- 
quired in t\\^ studcnL-prograni interface by the particular application under 
analysis. [f this type of accuracy Is uniniportant , the generator miglit 
select the terniinal with the touch panel < The same procedure could be 
followed for software, except that the mechanisms in thc^ implications 
analyzi--rs and the specifications generators would differ. As n features 
were each run through six cAI syptem elements, it is likely that many 
overlapping or redundant spec if ication.s would be generated* These would 
be collected nnd reduced to the shortest complete list* It can be argued 
that Figure 6 Illustrates the general operations involved in generating 
CAI specifications from task analysis inforoaation, but, in practice, the 
^inalyst will find that he need not go "hrough every single step that is 
implied by this diagram* There already are general specifications for PSO 
CAI that serve to structure and to simplify the task of generating specifi- 
cations for a particular application, PSO CAI is based on certain hardware 
capabilities that can be assumed to be available for any application: 

1 , Interactive, dynamic (animated) computer graphics * 

2 * Input -output inter fac ing for computer- control led front-panel 
simulators, voice synthesizers, and image projectors* 

3. Light pen (or the equivalent) and keyboard for student response 
inputs. It also must be possible to interface with the computer 
special response-input devices, e*g*, control sticks, and analog 
and digital controls and switches on man-machine interfaces, 

A* Computer-controlled peripherals for producing hard copy and for 
bulk storage* (The most economical hardware currently available 
are teletypes £or hard*copy and £loppy disks £or storage*) 

5. CPU cycle-time sufficiently fast to permit real-time simulation 
Involving accepting and responding to student inputs, and up^^ 
dating multiple variables within one "£rame" or one refresh cycle* 

6, Complete CAI system integrated into efi>ch terminal. 

All of these capabilities will be In the '*CAI testbed" hardware system 
as illustrated in Figure 7, being developed under Office of Naval Research 
£unding. 

The problem for generating hardware specifications for specific applica- 
tions, given the above j^eneral capabilities, is to identi£y the additional, 
special man-machine interfaces these applications would require, since only 
these would have to be designed for each application. Surface-structure 
features would be used to identify these special man-machine inter£ace 
requirements , 

PSO CAI alsQ is based on a certain general structure for the part o£ the 
CAI system, which is most of it, that is implemented by software; i*e*, by 
computer programs. The software capabilities generally required for PSO CAI 
can be characterized as follows: 
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Figure 6. Tentative Outline of Procedure for Generating CAI System Specifications 
from an Analysis of the Performance to be Taught 
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!• Unique interaction with each studout, via dynamic computer graphics 
and/or special studont-program interfacer^. The computer program 
must accept the' student*s input, calculate the values o£ variables 
required to respond to this input, and display these values in 
suitable format to the student. 

2. Continuous updating of tUe simulatu^d performance environment that 
is the subje^ct of the training. (In the case of the RIO trainer, 
i^hich -w&s a real-tittie problem, this updating occurred every tenth 
of a second and every time a student responded.) 

3. Continuous response-by-response tracking of student progress. 
This is necessary to generate the interaction with the student, 
to provide knowledge of results, and to provide data for student 
records . 

A. Instructional functions that can be placed either under student or 
under program control. 

5, Response-sensitive adaptive control over the instructional sequence. 
Currently, trlals-to-crlterion logic Is used. (The possibilities for 
using dynamic programming are under investigation.) 

6, Multiple response-variable recording and analysis. The use of multi- 
ple dependent variables provides needed information about what was 
going on during learning. 

7, Generation of the instructional sequence from program logic operat- 
ing on simple data-^struc tures , 

There are several techniques that might be considered for reducing the 
amount of programming required for new applications of PSO CAI^ Standardlza-* 
tlon of operations described for the five CAI system elements (see Figure 
2) might allow some of the same subroutines to be used across different 
applications by passing the specific values of variables to these subroutines. 

Standardization would necessarily extend to conventions for da^a-'Struc- 
tures as well. Programs and the data-structures on which they operate are 
so closely interrelated as to require joint design. Standardization also 
would include a system executive routine to manage the other programs. 
Typically such an executive program cycles through a list of "things to be 
done'* within some time frame, accepts certain interrupts from the external 
world, calls subroutines to service the Interrupts, and returns to the 
cycle. However, it is likely that this executive would be a fairly short 
program, and thus would not contribute a great deal to reducing the program- 
ming burden. 

Modularization of programs would make It possible to separate subroutines 
that perform general operations from programs that perform operations speci^ 
fic to an application. Modularization is a fundamental requirement for any 
system of programs; for convenient revisions, for growth, for using tech- 
niques such as overlay or virtual memory, and for more effective documenta- 
tion. 
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Writing iaterpreters for int^tructors to use for constructing computer 
graphics and dfi ta-s true tures , both of whicli must be specific to an applica- 
tiotij wouUl allow instriictorB to do tliis relatively quickly. The interpreter 
taku!} instructions written 1q a higher-level language (source code) and 
transiati^s (interprets) them into machine language (object code) as they 
art? i^xecuttKl by the computer. Thus, ati interpreter provides a more con- 
venient and faster method for constructing data-structures, which would 
be specific Co the application. 

We have produced an interpreter for computer graphics that allows the 
in^^tructor to construct figures by using function keys on the graphics 
terminal keyboard. The interpretive program interprets these function 
keys by writing subroutines to display the figures. An editor is included 
In the interpreter to allow changes and corrections to be made to the 
figure while it is being composed* The resulting display subroutines are 
stored in core, where they may be accessed by other programs (Rigney and 
Towne, in press) * 

We also have produced an interpreter for constructing data-structures 
for tlie TASRTEACH troubleshooting programs. The instructor need only 
identify and list inputs and outputs of each circuit in a device, using a 
numerical code* The interpreter then will reconstruct the entire network 
and generate from it certain types of relational information* 

This experience leads us to conclude that it is extreti^ly worthwhile 
to develop interpreters for CAI, since they save an enormous aonount of 
man hours otherwise necessary. We see no reasons vhy interpreters could not 
be developed for other areas in CAI as well* 

finally, a compiler for some standard programming language; e*g*, 
Fortran IV or Basic, would, of course, expedite producing all the computer 
programs, at the expense o£ possibly using a little more memory for the 
object code. At the present time, only an assembler/editor is available 
for the minicomputer used in the PSO CAI hardware system. 

The big question is just how far genf!rali2ation techniques can be 
pushed to develop what might be called a general operating system for PSO 
CAI. The successes? we have had in doing this for limited areas have also 
taught us that this Is far from a trivial problem. Nevertheless, the enor- 
mous amount of labor required to produce instructional programs for the 
alternative, "ad hoc frame-oriented" (Carbonell, 1972) CAI, is, we believe, 
unacceptable in the long run. The problem will be given more intensive 
consideration later* Meanwhile, in the following sections we will turn 
our attention to procedures for generating soici^are specifications* 



l^xampies of Procedures for Generat inp. Software Specifications 

In the ca«e of specifications Cor software, it has been our experience 
that producing an instructioaal sequence flowchart serves to force the 
analy.sis in the right directions;, and results in a means for communicating 
the basic structure of the instructional sequence to the computer programmers. 
This Is, we hasten to add, an iterative process. When computer programming 
begins, it uncovers loose ends that were left unresolved or that were overlooked 
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that now must be finished. We emphat?i2e that the instructional sequence 
flowchart is the primary outcome ol the task analysis, and is the principal 
medium for defining and communicating ^ipecif ications. It will be the 
source of specifications for stimulus and response features, concent media-' 
tors, L & M mediators, knowledge of results, and £or their sequential 
organization* 

The task analysis to identify surface structures and content mediators, 
and to select L 6c M mediators, must be summarized in this instructional 
flowchart* This is why we find that the products of the current fad in 
military training for painstakingly writing out long listd o£ behavioral 
objectives are not very useful^ We surveyed this field recently (see 
Appendix B) and concluded, reluctantly, that these analyses are simply too 
general, leaving out details that are essential* It is necessary to do an 
analysis that will produce the exact information needed for this purpose* 
An example of a reasonably detailed task analysts for the Rio's job ^n air 
intercepts is reproduced below* This exf.mple includes the analysis only for 
the first of four major goals during the Intercept (1) establish collision 
course, (2) establish displacement distance, (3) primary attack, and (4) 
reatt^ck* However, it illustrate*^ about the right level of detail for this 
type of analysis * 



Activity List; Goal 1: Closure (Collision Course) 



GO^L 1: Establish a **best closure** on 
any designated target* 
(Co-speed) 

Action 1: 

Receive Bogey data from GCI Controller 
Verbal (Radio) * 

a» Bogey Bearing (BB) Degrees True 

b. Bogey Range (BR) miles 

c. Bogey Heading (BH) Degrees True 

Action 2\ 

Determine Reciprocal of Bogey Heading 
(BHR) to fix initial values of the 
intercept triangle (Figure 1). 

a. If BH< 180^ True 
BHR = BH + 180** 

b. If BH>180^ True 
BHR = BH - 180** 

Action 3: 

From BiiR and BB determine Target 

A.-pect Angle (TA) • 

a- If BB right of BHR TA right 

b. If BB left of BHR TA left 

c. TA = Difference in degrees between 
BHR and BB 



1» Alphanumeric display 
2. Additional Data: Fighter 
Heading (FH) 



Errors: miscomputes BHR 



Errors \ 

K Wrong value of TA 
2. TA determined in wrong 
direction 
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Ac t: ton 4 : 

From Fl^',hter Heading (FH) and Wo^^K^y 
liearing (liB) determine Angle Off 
(AO), Label AO *^right** or *'left/' 

a. tf f^B< PH; AO **left'' 

b, If BE> t*H; AO ''right** 



Action 5: 

Fighter position in relation to Bogey 
flight path, 

a. If TA and AO bo ^h '*left*' or 
'"right" fighter is across Bogey 
flight path 

b. If TA and AO different, fighter 
closing on Bogey flight path 



Action 6; 

Determine aircraft (fighter) **best** 
initial closure; course; collision 
course (CO) 

a, CollLRion Course = that fighter 
heading where TA ^ AO 

b. Find difference between TA and AO 



Action 7: 

Turn fighter to Collision Course, 

a. If AO *'left," turn left 

b. If AO **right," turn right 



Action 8; 

rJetermino number of Degrees To Go to 
Bogoy Heading 

a, DTG - Difference FH and BH 



Angle Off in degrees: the bearing 
of tJie Bogey relative to the 
longitudinal axis of the Fighter, 
measured in degrees left or right 
of the nose* 

Errors : 

1» Wrong value of AO 

2» Wrong direction of AO 



Fighter across Bogey flight path 
(heading away from Bogey), separa- 
tion between the two A/C is 
increasing. 

Errors: If "labels** (right/left) 
of TA and AO erroneously deter- 
mined, fighter position error, 
could cause turns in wrong direction* 



**Best*^ initial closure course is 
the ''collision course,'* i»e», that 
fighter heading that would cause 
the two A/C to collide if at same 
altitude » 

Errors: determines wrong value of 
difference between TA/AO* 



Errors : 

1, Turns wrong direction 

2* Tuvns wrong Uo* of degrees 

Proper directive terminology: 
*'Turn left/right (hard, easy), 
XX to XXX*^ 

XX; No* of degrees of turn, 

Exp, 20, 30, etc* 
XXX: Fighter Heading in degrees, 

Exp, 350, 270, etc* 



Degrees To Go (DTG): the number 
of degrees of difference between 
Fighter Heading and Bogey Heading; 
the numbers of degrees Fighter 
must turn (left or right) to 
parallel Bogey Heading* 
(This action could also be performed 
prior to turn to CC and would be 
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Action 9: 

Wltli Figliter steady on new (Collision) 
Heading: 

a. Give ARE O Report 
b» Give dTG 

c. Test for Rood CC establishment; 
a collision course is good if 
BB/AO holds a constant value* 



Action 10: 

Corrects collision course 



determined again after turn to CC) 



AREO: A report to pilot to pro- 
vide information ou Bogey* 
A = Angle Off 
R * Range 

E = Bogey Elevation in feet above/ 

below Flight Altitude (if known) 
0 " Overtake (closure) if known 

Test for CC: 

1, Bogey not yet acquired on fight" 
Gr radar: 

a* Receives Bogey Bearing 
from GCI, determine AO. 
(1) If AO increasing/ 
decreasing, CC not 
adequate * 

Errors : 

1* AREO Report errors in value or 

terminology 
2» DTC value 

3. Fails to detect poor CC 
establishment 



Repeats Action 6, 7 
If bearing change small may shift 
to Goal 2j and acquire Bogey 
on radar » 



A« we pointed out above, producing detailed specifications for computer 
programs is an iterative process. The first flowcharts will be rather 
general, but they will be useful for sustaining the dialog between the 
instructional technologist and the programmer. The main requirement is to 
put down the flow of the different events that are planned for the instruc-* 
tional sequence, in term^s of major events and decision boxes. Then, this 
crude model can be analyzed into final detail. An example of a preliminary 
flowchart for jne phase of the RIO*s job in air intercepts is given in 
Figure 8, During this process, many revisions will be necessary^ since some 
planned displays or operations may be infeasible to program, or may be in 
error, or better ways of accomplishing certain objectives may be thought of 
during discussions* Thus, the instruction flowchart is a medium of communi-* 
cation between two different technologists, and as such, should remain fluid. 
It would be a serious mistake to try to produce one, grand, detailed, final 
instructional flowchart before ''calling in" the programmers. Ultimately, some* 
thing like the flowchart reproduced in Figure 9. will evolve. By this time, it 
will be more like documentation of what has been done than a plan for what is 
to be done* It will reflect the Interaction between instructional technologist 
and programmer by containing many terms characteristically produced by pro- 
grammers. Figure 9 reproduces the complete flowchart, at a rather general level, 
considering a programmer's needs, for the RIO intercept training computer program, 
as it was finally evolved^ 
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Legend of Variables : 

Speed = J: Varies over the range 220 to 500 knots* Initialized at 220 
kts, Increases by increments of AO kts determined by student 
performance* As criterion performance is achieved, speed is 
incremented and further practice is required until the 500 kts 
level Is reached* The Test problems are performed at this 
speed level* At speeds of 300 knots and above both Sparrow 
and Sidewinder are fired* 

Mode Number = K: Utilized to branch the student through the primary practice 
modes: btode 0 = Free Fly, Mode 1 = Static Practice, Mode 2 ~ 
Dynamic Practice* 

Target Aspect Catagory (TAG) = L: Utilized to select problems of a given 

TAG* Since Target Aspect is a variable which determines the 
degree of maneuvering and technique required In an Intercept, 
TAG is arbitrarily divided into Low, Medium, and High cate- 
gories* The instructional strategy requires that the student 
perform an equal number of intercepts in each category to meet 
performance criteria* 

TOTE: The displayed list of computational variables that the student 

must determine and input to the program prior to commencing an 
intercept. Static Mode is the practice module which develops 
speed and accuracy in determining correct TOTE values* 
Depression of the TOTK Key during and intercept will cause 
display of current values of all intercept variables. 

TRI: Intercept Triangle: A plan view of the basic triangle formed 

by the intersection of Bogey course line. Fighter Gourse line, 
and the line of Bogey Bearing from the Fighter* In Static 
Mode the triangle is static (the fighter and Bogey positions 
are fixed)* In Dynamic bfode the triangle is updated in real 
time representing the relative positions of the Bogey and 
Fighter as the intercept progresses* The triangle is displayed 
automatically if a TOTE error occurs or on demand when the TRI 
Key is held depressed. 

Pause; A student-^controlled program function* Depressing the Pause 

Key effectively **free2cs" the problem allowing the student 
to evaluate elements of the intercept* A second depression 
continues the problem from the instant of pausing* 

pllit: Hit probability - a numerical score determined for both the 

Sparrow and Sidewinder missiles by arbitrary algorithm- This 
number Is an index of the student proximity to optimal raage 
angle off the target, and turn condition at the time of weapon 
launch. A hit probability of -80 for the Sidewinder and »50 
for the Sparrow must be achieved to attain criterion performance* 

Latency: The time required to perform an intercept function* Latency is 

measured between responses on TOTE, for the overall completion of 
the TOTE list (total latency)* A latency of 60 seconds is 
established as the criterion for TOTE perf ormaace * 
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Student-Program Interface : Stimulus Display > For computer graphics, 
features for tUe stimulus display must be specified In terms of what is to 
be unchanging and what is to change on the display* The course starts with 
a certain type of display* Features of this change in relation to the stu' 
dent*s responses, under the control of the Instructional-Sequence Scheduler* 
What migiit be called the cross-ftectional and the longitudinal features of 
the stimulus display must be identified* This is best done in terms of dia- 
grams of all the different display configurations, in which each feature is 
drawa to scale as it is to appear, and is integrated with the instructional 
flowchart that shows details of the sequencing logic for the different 
displays * 

The programmer's flowcharts for computer-graphics programs can be con* 
structed from these diagrams and the instructional flowcharts, although as 
always, several revisions of flowcharts and much verbal interaction between 
the instructional technologist and the computer programmer will be required* 
The display generator programs will necessarily interface with, or will be 
subroutines in the program for generating the interaction with the student, 
which accepts student inputs that will influence the composition of the dis- 
plays* An example of a progranmier 's flowchart for simulating a radar B-scan 
graphically is given in Figure 10* It will be seen that this is aL a finer 
level of detail than the instructional flowchart reproduced in Figure 9* 
This will always be the case* 

Computer graphics are drawn by subroutines which run in a display pro- 
cessor computer, and which are made up of special display instructions* Com- 
puter graphics typically require a lot of core, since a great many computer 
words are needed to draw even a moderately complicated figure* 

Specifications for special inteifaces; e*g., front panels of operational 
equipment, would be determined from the instructional flowchart in a manner 
similar to the derivation of specifications for computer graphics* Each 
display and each control on the front panel must be identified and its role 
in the instructional sequence must be described. This involves identifying 
different states of the device that are established by different configurations 
of front panel controls, and describing these states and conditions in a 
data structure somewhat analogous to the data structure for computer graphics. 

These special interfaces also require that appropriate hardware be de- 
signed for representing the interface and for attaching it to the computer* 
The 10 interface and the A/D converter multiplexer, indicated in Figure 7, 
are examples of this type of tiardware* This generally can be relatively 
simple, partly because humans respond at relatively low data-rates* A good 
bit of software; is required to make this hardware work; transmission of 
information back and forth must be under program control. 

Student -Program Interface : Response Recording . Student responses may 
be made via a keyboard, light-pen, or special control on a special man-machine 
panel. In all cases, the computer program looks in an input register to 
find the response information. Student inputs in raw or processed form will 
be used by the instructional sequence (IS) generator, scheduler, and optimizer* 
This IS generator uses them as part of the information needed for controlling 
the immediate student-program interaction. 
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XLOC = XLOC +AX 



DEFINE X LOCATION 
OF B-SCAN USING 
XLOC 



STORE DISPUY 
INSTRUCTION IN 
DISPLAY LIST 



RETURN 




Variables ; 

RTLIM = Right Limit of B-Scan 

LTLIM = Left Limit of E-Scan 

AX = i 5 =■ Movement of B-Scan During One Cycle (1/40 Second) 

XLOC = X Location of B-Scan 



Figure 10* 3-Scan Flow Diagram 

(The B-scan is simulated by moving it 5 rasters (5/128 inch) each 1/40 
second* With this frequency, the eye cannot distinguish discrete movement 
from continuous motion.) 



-28- 



MAVTRAEQUIPCEN 73-C-0065-1 



\ 

Student responses also are recorded* and which responses to record 
constitutes an Important decision. An enormous atnount of response and 
response-related data can be recorded by a minicomputer. Since these data 
must be analyzed by computer programs, and statistical analysis programs 
tead to be complicated and tedious to write, a problem Is to determine th^^ 
minimum amount of student response data that is necessary to record for 
analysis . 

A program to record responses must code response categories in such a 
way that they can be kept separate during processing, take care of any 
temporary buffering of output* and pass the data to the program controlling 
recording on disk or otber medium* 

If values of large numbers o£ response variables are to be recorded, 
coding to identify variables can become fairly complex^ as Appendix A shows. 
All of the response variable^ shown there were originally defined from an 
analysis of the man'-machin<3 Interface interactions in the RXO^s Air Inter- 
cept individual skills trainer* There are instructional features unique 
to the trainer to which the student must respond. 

Student jPa^ a: Student SHC * Recall the definition of a student suffi- 
cient history, given by Atkinsoa and Paulson (see page 6)» There must be 
a program to compute this sufficient history for whatever purpose it is 
needed* As we illustrated above, the need may be for input to an optimiza- 
tion program, or it may be to satisfy training managers* or it may be for 
data for a research design* It is clear that there could be a variety of 
different sufficient histories, so the problems for generating specifications 
are to decide what the need is, and then how to compute the sufficient his- 
tories that are required* 

Computing a student sufficient history for use by an optimization model 
is a complicated matter, as Atkinson and Paulson (1972) and Chant and Atkinson 
(1973) have made clear* For an illustration of another approach see Wollmer 
(1973)* 

We emphasize that several sufficient histories may be needed for each 
student. Descriptive analysis of '*what went on" during a field L:rial;, for 
example, may require that values be recorded for a large number of variables, 
whereas an adaptive controller may need* instead i some summary index* For 
example, the trials-to-criterion logic we have implemented in the RIO trainer 
used a "student word** and looked at that word only while the student was 
engaged by the system (see Figure 11)* When the student was transitioned 
to the next level, the word was cleared, since the need for this information 
was only temporary* 

Student Data : Student Records . As indicated in the CAl system ele- 
ments diagram (see Figure 2), student records contain those student suffi- 
cient histories that are needed for more than temporary purposes* They are 
stored In a memory until they are needed either by the Adaptive Controller 
or the Statistical Analyzer. Student records ^uld most likely be stored 
on disk files* A temporary expedient, which has been used for the RIO train- 
er, is to punch them on paper tape for later analysis, but this Is in the 
category of ''what to do until the disk arrives*" The slow punching and 
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Level Indicator (0-7) 
Bit 0 = l>Static 



TA Category Indicator 
(1-3) 

Bit 10 = l^Success on 
Previous PB 
= O^Failure 



^ Student Progress in TA 
Category 



-> Student Number 



Bit 0 = l*Static Student 
Mode 

0>Dynafiiic Student 
Mode 



Figure 11, Example o£ a student sufficient history used for trials-to- 
criterion logic. Information in the word is compared for a 
match with criteria for transitioning to the next level 
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slow reading of paper tape, combined with the possibilities for mislabel- 
ling, physical damage, or getting lost in the mail, make this an unreliable 
and expensive process. Storing student data on a disk file requires a disk- 
controller and a data-formatter. These operations are performed partly by 
hardware and partly by software, both general purpose. The programming re- 
quirement specific to a particular application is for a routine that will 
'^talk** to the general program and pass data to it in strings that will fit a 
general format; e.g., a record length, and that will read records from disk 
to core when they are required. 

In the RIO trainer, the data-capture routine sends data to a teletype 
paper tape. It sends data after the static mode and after the fire display 
for the dynamic mode. In order to maintain the sequential continuity of 
the problem, data is first stored in a double-buffer, and then dumped dur- 
ing display cycles when the teletype is free. In this manner, the problem 
continues running while data is being stored on paper tape. 

The appropriate criterion (static or dynamic) subroutine is called by 
the RIO program; then the data capture subroutine is called from the cri* 
terion check subroutine. There are two lists of pointers, one each for the 
static and dynamic phase data. When called, this subroutine accesses the 
words indicated by the pointers, and uses a subroutine to store the word 
in a teletype compatible-formaL in the teletype buffer. Dumping the buffer 
occurs automatically; when filled, it will start sending data to the tele- 
type until it is emptied, at which time the dumping routine waits until toore 
data is stored in the buffer. 

Adaptive Controller : IS Scheduler ^ Whether or not the IS Scheduler is 
a computer program separate from the IS Generator (and IS Optimizer) depends 
on a number of factors. Scheduling, that is, organizing problems or "frames" 
into a sequence can be the result of, or the output of these other functions. 
Whether control over the sequence resides in the student, in the CAI system, 
or in both influences this. In PSO CAI, a mixed-initiative form of control 
is used which is truly response-sensitive. That is, the fine-grained instruc- 
tional sequence is unique to each student. It is convenient for the scheduler 
to be an executive that controls the generator routines « However, in forms 
of adaptive control such as dynamic programming, that are basically norm- 
referenced control, it might be more convenient to combine the scheduler with 
the optimizer. 

It is worth noting that an important part of IS scheduling is keeping 
track of where the student is. In a course of any length, students will 
not be able to finish in one session. Any one terminal will be used by a 
number of different students. There must be logic to identify who is on the 
terminal, to start the studervt again at the right place, and to store **where 
he is'* information continuously, so that unscheduled interruptions will 
not require that the student start all over at the beginning. Some of this 
information would be passed to the student data programs. 

Most PSO-CAI IS Scheduler prograrns cycle through a "loop" of things to 
do, since PSO CAI is concerned with giving students practice in performing 
certain well-defined tasks or problems. The IS Scheduler under these circum- 
stances is the top-level executive program. An example of an IS Scheduler, 



ERLC 



-31- 




NAVTRAEQUIPCEN 73-C-0065-1 



* 

COMPUTE TRI 



X 

CALC OPTI- 
MAL PATH 





INCR. SEC 
& SECT. 



CYC = 0 



INCR. 
DELAY 



DISPLAY 
TRIANGLE 



NO 
(DELAY 
0) 



IS 
DELAY 
OVER? 



(DELAY < 0) 



Figure 12. The RIO Trainer IS Scheduler Programmer's Flowchart 
(Delay Is a counter, not a tlmer^ that uses a 
special Instruction) 
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for the RiO trainer^ illustrated by the following programmer ^ a flowchart 
(Figure 12). 

Adaptive Controller : IS Generator. We assume that the IS Generator 
is a program that creates the "within problem" Interaction with the student 
while he is performing, and that the instructional sequence has not be pre- 
determined in the form of programmed Instruction by an army of course authors. 
This generator contains the logic for interacting with the student, which 
must necessarily include calling subroutines or other programs that generate 
stimulus displays or record responses. 

How general can the IS Generator be? Wc have developed and demonstrated 
IS Generators for a class of problem^solving activities called "trouble^ 
shooting" (Rigney, et al., 1972a). Limits of this class have not been ex- 
plored, since there has only been time to develop data*structures for elec- 
tronic and electromechanical devices. We see no particular reason why the 
class could not include medical "troubleshooting;" i.e., diagnostic activi- 
ties for animate as well as for inanimate devices. We do know these IS Gen- 
erators, in their present form, are restricted to tasks in which the student 
must perform information-sampling operations on some malfunctioning system^ 
interpret the information from these tests, and revise his hypothesis set 
about possible causes of the otalfunction until, by successive approximations, 
he finds the actual cause. 

We Also have developed and demonstrated an IS Generator that is general 
to a class of procedural tasks which are concerned with changing the states 
of devices by manipulating inputs to the devices. Again, the boundaries of 
the class have not been explored. The general requirement is that the struc* 
ture of the task the student is to learn be representable in the form of a 
final goal, and intermediate goals, linked to each other and to actions to 
be performed to accomplish goals, by any of a group of logical operators which 
define the task-structural relationships (Rigney, et al., 1972a). 

These programs have taught us certain facts about generality requirements. 
First;, there must be some general task-structure that is to be the basis for 
interaction between the program and the student. A general structure can be 
defined for troubleshooting, and for procedural tasks. Second, there must 
be some general way of representing states of devices being "worked-on," 
of student progress, of problems and procedures so that these states can 
be continuously updated as they change. This requires also that it be pos- 
sible to represent relationshipf; among events, conditions affecting states, 
and states . 

Third, it must be possible to describe essential features of specific 
devices and specific tasks iu terms of general data-structure formats and 
codes. The program then can operate on these d.^ta structures in the same 
viays , regardless of the particular device or tasks involved, until it com- 
municates with the external world; i.e., the student, when it then takes 
the additional step of displaying a specific lexical word or phrase that is 
matched to an internal code. 

fourth, the student mufst inform the program of what he is trying to do. 
This amounts to the student telling the program what his next goal is going 
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to be. In troubleshooting, this would be the next problem he is going 

to take, or the next indicator he is going to sample for symptom information* 

Knowing what goal the student is working toward allows the program to 

track his progress toward that goal^ offer resistances and provide progress 

information. 

It Is important to appreciate the complexity of the student- program 
interaction that is required for PSO CAI» This is no multiple-choice 
question game. The programmer's flowchart for two of the programs in the 
IS Gen(?rator for the RIO trainer are illustrated in Figure 13* 
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Figure 13. Programmer's Flowchart for the 

Command Interpreter (Sheet 1 of ^) 
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Figure 13. Programmer's Flowchart for the 

Command Interpreter (Sheet 2 of 4) 
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Legend: 

The CpMM program is "called'* by the IS Scheduler illustrated in 
Figure 12. The COMM program accepts commands and other inputs from the 
keyboard* These student inputs are described as follows: 



DIGITS 



R and L 



CR 



ANS 



ESC 



TRI 



Numerical values are entered for the toteboard (static) 
phase, and for the definition of turns in the dynamic 
phase. If a number is entered immediately after the 
type and severity of a turn is defined, it represents 
the number of degrees to turn. If the '*T0" key is used, 
then the number represents the heading at which the 
fighter should complete the turn. 

Th^se keys are used to define right or left for the input 
of TA, MUA and AO. 

Carriage return is used to define a completed entry either 
for the toteboard or the definition of a turn. 

The answer key is used only during the static phase (filling 
in the toteboard). If the student cannot solve the prob- 
lem for a given entry, then the answer key will provide 
the correct answer (the student must have tried at least 
one entry) . 

The escape key is used to erase an incorrect entry. It 
does not penalize the student if he makes a keyboard error 
and then uses the escape key before hitting carriage return. 
For a turn entry, the escape key will clear the turn dis- 
play and level the aircraft* 

The triangle displays the geometric "birds eye" view of 
the problem on the top half of the CRT. It includes the 
optimal path and a trace of the bogey's path on the B-scan 
display. 



TOT 



PAUSE 



TURN COMMAND 



TO 



The toteboard displays the current toteboard values. It can 
be used only during the dynamic phase of the problem. 

The pause key stops the motion of the fighter and bogey, 
thus ''freeing'* the problem until the pause key is depressed 
again. The key can be used only during the dynamic phase 
of the problem* 

There are eight turn commands, four each for port and star- 
board. They specify four levels of severity (easy, stan- 
dard, hard, and hard-as-possible). The turn command also 
puts the program in a mode to receive the numerical input 
specifying the parameters of the turn. 

The TO command is used immediately following a turn command. 
It is used to define the heading to which the RIO wants 
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EAS 



TTN 
STOY 

CENG SCALE 



LOCK 



the aircraft Co tura. I£ this cominaad is aot used, the 
number input ^ill be Interpreted as the number of degrees 
the aircraft is to turn be£ore leveling. 

The TURN 360 key used to turn the aircraft 360*^ , leav- 
ing the aircraft at its present heading a£ter the turn. 
Typically, this turn is terminated by a STDY command. 

The EASE key reduces the severity of a turn one level (e.g., 
a standard turn reduces to an easy turn, an easy turn 
reduces to level flight). I£ the aircraft is not in a 
turn, the EASE key does nothing. 

The TIGHTER key increases the severity of the aircra£t 
turn, unless the aircraft is either in a hard-as-possible 
turn or is not turning. In this case this key does nothing. 

The STEADY command stops a turn and levels the aircraft. 

The CHANGE SCALE affects the B-scan range. The range ini- 
tially starts at 40 miles range, but it can be switched to 
20 miles. This key changes the range o£ the radar from 
40 to 20 or from 20 to 40 when it is depressed. ^ 

These two keys simulate the ability to control the radar 
scan manually. The scan is in the direction o£ the arrow 
at a slow rate. 

The LOCK function occurs when the bogey is on the B-scan 
display and one of the two keys was just released within 
3^of the bogey. This displays: the ROC (rate of closure) 
circle and keeps illuminating the target until it leaves 
the range and angle of£ limits of the B-scan. 



SECTOR SCAN 



SFrf FIRE 



SW FIRE 



There are three areas of sector scan: Left (43 le£t - 3 
right); Center (23** left - 23*^ right); Right (S^ left - 
45° right). When the scan is so restricted, it naturally 
paints the bogey more frequently. 

The SPARROW FIRE key simulates the firing of a Sparrow 
Missile. The display list £or the fire display is defined 
by the computer and stored until the sidewinder is fired. 
The indication FOX 1 Is displayed on the screen to signi£y 
that the Sparrow has been fired. 

The SIDEWINDER FIRE key simulates the firing of a Sidewinder 
Missile, terminates the problem, and presents a fire dis- 
play for the Sparrow (if fired), and Sidewinder Missile. 
The criteria for success or failure is evaluated depending 
upon speed level and the hit probabilities of the two 
missile firings. When the fire key is depressed again, the 
computer selects the next appropriate problem for the student. 
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The FREE FLY key is available only when the fire display 
is present* It gives the students the problem previously 
completed In tho free fly mode* 

The STOP key is used when the student has completed a 
session; the teletype buffer is dumped on paper tape, and 
leader is punched allowing the operator sufficient room 
to appropriately label and tear off the tape* 
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The COMP program (see Figures 12 and 14) does the computations 
necessary to '*run'' the intercept problems* Basically, this is a matter 
o£ updating the positions of the bogey and fighter on the CRT, The values 
of variables that are computed by COMP are used in the computer graphics 
programs to generate the real-time displays, and for tracking the student's 
progress, so that his values o£ intercept triangle variables and his fir- 
ing positions can be scored right or wrong* 

The subroutine CO^[P computes the angular relationships, distance rela-^ 
tionships, and calls the subroutine to compute the rate of closure for the 
fighter and the bogey. This requires the X and Y position of the two air- 
craft, and both headings. The computation routine is strictly a static rou- 
tine, it calculates X and Y, which provides sufficient information to deter- 
mine the bogey bearing with the use of a table of tangents. Given bogey 
bearing and the aircraft headings, all angular relationships can be calculated. 
Range and displacement distance is calculated using X, Y, computed angu* 
lar relationships, and a sine^cosine table. 

Motion is simulated by moving the X and Y locations of the fighter and 
bogey as appropriate for their speed and direction. Their positions are 
updated 6»very 1/10 second, at which time the COMP subroutine is tun to keep 
all angular and distance values current. It has been determined that greater 
frequency of updating would not noticeably improve the fidelity of the simula- 
tion. 

Adaptive Controller : IS Optimizer . Aside from Atkinson's group we 
know of no day-to-day applications of quantitative process-control models 
in CAI. We are using relatively crude, multi-stage trials- to-criterion 
logic Cor adapting to individual differences. This has been demonstrated to 
be effective, in the sense that more able students require less practice to 
finish the course than less able students, and the group variances are 
reduced to a very narrow range at the end of the course (Rigney, et al., 
1973b). 

The logic for doing this is a part of the IS Generator-Scheduler programs. 
Since it Is relatively simple, there is no particular reason to separate it. 
Criteria for latency, errors, and accuracy can be . ^t. Student words are 
updated with values of these variables as the student works. The IS Optimizer 
logic looks at this student sufficient history after each problem (for 
example, see Figure 9). When the values in the words match the criteria, 
the student is transitioned to the next level. 

A Trial*to*Cr iter ion Strategy for the RIO Trainer . This strategy requires 
the student to continue attempting practice problems until he has satisfied 
the criterion to advance to the next level, repeating this process in new 
categories until the course is completed. The basic criterion is to com- 
plete two consecutive problems in each target aspect category (TAC) success- 
fully without the use of any of the instructional aids available on key 
demand. 

The levels are structured as follows: 

1. The first level is an introductory level where the student is given 
one free fly problem at each of 3 target aspect categories. 
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Figure 14» Programmer's Flowchart for the COMP Subroutine 
(Sheet 1 of 2) 
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2. The next two levels are static problem levels* The first static 
level displays the triangle while the student fills in the tote- 
board. In the second static level the triangle is eliminated* 
In each level, the student must complete two consecutive problems 
without error in 60 seconds or less for each of 3 target aspect 
categories » 

3» The next two levels are dynamic problems at a speed of 220 and 260 
knots, respectively* The criteria for these two levels are to 
achieve a hit probability of *80 or better for the Sidewinder fire; 
a Sparrow fire is not required* The toteboard must be completed 
without error* Each of three target aspect categories begins with 
a free fly problem* 

4* The next five levels are dynamic problems at 300, 340, 380, 420, and 
460 knots, respectively* The criteria for success include that of 
220 and 260 speed levels and also require a Sparrow hit probability 
of *50 or better* Each level has 3 target aspect categories* 

5* The last level is the transfer test* It includes 20 problems which 
are run at 500 knots* The target aspects of these problems include 
low target aspect problems and problems with target aspects not 
used for the three target aspect categories for practice problems* 
The student has no aids available, and advances to the next problem 
regardless of the results of previous problertis* The conclusion of 
these problems is the end of this course* 

Content/Mediator Files * These are the data-structures the IS Generator- 
Scheduler operates upon to produce the instructional sequence* For PSO CAI, 
they tend to be fairly simple lists of specifications describing selected 
features of devices, tasks, or problems* Although the formats of these data*- 
structures can be general, as indeed they must be for the IS Generators 
discussed above, their contents obviously must be specific to the application* 

As we said before, it is feasible to produce interpretive programs to 
allow no n- programmer personnel to prepare data-structures* Otherwise, a 
team of a subject-matter expert and a computer prograoHiier must be formed* 
Since the subject-matter expert is likely to view the wrld in an entirely 
different way than the computer programmer, a certain amount of discussion 
will be required to extract the needed information* 

In the case of the RIO trainer^ many practice Intercept problems were 
generated from a basic problem, using an instruction to the program having 
the following format: 
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Pointer to the base problem 
^ (0 - 6^) 



^Fighter Heading deviation 
(0 - 43) 

^1 > rotate about X-axis 

0 > do not rotate about X-axis 

1 > rotate about Y-axis 

0 > do not rotate about Y-axis 



Figure 15 » The problem pointer word used in generating 
practice problems 



There are 11 basic problems for each tsrget aspect category* Each prob- 
lem may be rotated about the X-axis, Y-axis, or both* These base problems 
define the initial X and Y positions of the fighter and bogey and their 
headings* The fighter is set on a collision course for the basic problems* 

The problem pointers specify the fighter heading change, the base prob- 
lem, and the rotation* The deviation from collision course is a variable 
that affects the difficulty of the B^scan interpretation after the toteboard 
is completed. A deviation of 90*^ or 270*^ starts the fighter perpendicular 
to the collision course, causing the target aspect to change rapidly while 
the student is entering toteboard values and subsequently waiting for the 
fighter to reach collision course* This variable is restricted for the 
first few speed levels, then varied more freely after the student has had 
a chance to familiarize himself with the procedures and the trainer* Prob- 
lems at the start of a target aspect level are given a higher deviation, as 
they are flown in free-fly and hopefully serve to illustrate the effects of 
drift while achieving collision course. 

There is a total o^ 128 practice problems for each target aspect, or a 
total of 384 problems for the 2 static and 7 dynamic levels* The 20 test 
problems have 20 base problems and 20 pointers* This is necessary as the 
characteristics of the transfer teat problems are more vigorously controlled 
than the practice problems* Also for the test problems, it was deemed more 
Important to consider opening or closing target aspect, and crossing the 
bogey flight path for lo^^ target aspect problems* 
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The purpose o£ oornputer-controlled problem selection is not only 
convenience of operation, but is primarily to reduc3 uncontrolled varia- 
tion for the statistical analysis of the problem* When each student starts 
a given target aspect category within a speed level, he will start with the 
same problem as every other student, in both "stripped** and "enhanced" 
versions of the trainer* Obviously, due to advancement based on a trials- 
tO"criterlon strategy, each student will require a different number of 
problems to satisfy the criterion* 

Instructional Sequence . Since in PSO CAI this Is "made up" by the 
Adaptive Controller operators, it literally does not exist until the moment 
of use. Its major constituents, content, knowledge of results, content 
mediators, and L M mediators, are produced by computer programs operating 
on data-structures and student Inputs. 

Statistical Analysis ^ These programs could be standardized to accept 
sufficient hls*;orles from the SHC programs. Once students-by-varlables 
arrays are available, standard statistical analysis programs could be used. 
Sums, sums of squares, and N's Cor individual students and for samples and' 
subsamples are useful intermediate products to have a data-summarizing pro- 
gram printout. These allow convenient visual inspection of the data. This 
may reveal Information or suggest analyses that may not have been foreseen. 

Where multiple dependent variables and multi-mode, multl-dlCClculty level 
instructional structures are used, programs to analyze data become relatively 
complicated. Computations are simple, but data must be partitioned into 
many different categories. The short word length of minicomputers; e.g. 
sixteen bits, is not convenient for accumulating sums and sums of squares, 
or for doing mul tlply-dlvlde. Floating point or multiple-precision logic 
Is required. The point we are rnaklng is that these data-processing programs 
require a substantial proportion of programmer time to produce. 
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SECTION IV 



SUMMARY AND CONCLUSIONS 



This report contains a description of FSO CAI, and general procedures 
to follow in :LmpLe[nencing PSO CAI. Key personnel arc siibject-tnatter 
experCs, instructional technologists, and computer programmer/analysts* 

The six elements required in any CAI system were described* Each of 
these must be represented in the design o£ a CAI system. Each, except the 
student element, contains operations that must be implemented either by 
hardware or by software* Some illustrations of how these operations were 
implemented in the RIO trainer are given^ in the form of examples of task 
analyses, instructional flowcharts, and of the computer programmer's flow-* 
charts that were derived from them* 

We considered how a **general operating system** of computer programs, 
and a general hardware system^ for all applications of FSO CAI might be 
produced. The most obvious general approaches to software production 
currently visible are the development of interpreters and compilers, which 
would permit more rapid production of computer graphics, computer programs, 
and data-structures* For teaching well-^def ined classes of job skills. Instruc- 
tional Sequence Generators, which constitute roughly fifty to ninety percent, 
depending on the amount of computer graphics, of the computer programs 
needed, can be general within a class of job skills- The lASKTEACH Instruc- 
tional Sequence Generator is an example (Rigney, et al*, 1972a)* Conditions 
necessary for this kind of is Generator are: 

1, There must be some general way for describing task-structures that 
can be the basis for interaction between the program and the stu- 
dent. General structures have been defined for troubleshooting, 
and for procedural tasks, 

2, There must be some general way of representing states of devices 
being 'Wrked-on," of student progress, of problems and of pro- 
cedures so that these states can be continuously updated. This 
requires also that it be possible to represent relationships among 
events affecting states^ and different states* 

3, It must be possible to describe essential features of specific 
devices and specific tasks in terms of general data-structure forimts 
and codes. The program then can operate on these data-structures 

in the same ways, regardless of the particular devices or tasks 
involved, until it communicates with the external world; i.e*, the 
student, when it then takes the additional step of displaying a 
special lexical word or phrase that is matched to a general internal 
code, 

4* The student must inform the program of what Ihe is trying to do. 

This amounts to the student telling the prograc^ what his next goal 
Is going to be. In troubleshooting, this w(?aild be the next problem 
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he is going Co Cake, or wichin a progrant| Che nexC indicaCor he 
is going Co sample for sympcom informacion, or che naxc fronc- 
panel CesCt drill he is going Co cake» Knowing whaC goal Che 
sCudenC is working coward allows Che program Co crack his progress 
Coward chaC goal, offer assisCance, and provide progress 
informacion* 

A general hardware sysCem for supporcing PSO CAI was described* WiCh 
Che excepclon of some 10 inCerfacing, iCs componenCs are o£f-Che-shel£ 
commercial iCems * 
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APPENDIX A 



RIO INITIAL DATA CAPTURE DEFINITION 





No 


Digits 




1 




12 


PP 


2 


A 


12 


PH 


2 


n 


12 


IT O 


4 


c 


12 


PE 


5 


E 


12 


SE 


V 




12 




7 


G 


12 


SH 


8 


H 


12 




9 


D 


9 


Turn S60 


10 


T 


1 2 

J. 


Turn Stn ^ he&d inc^ 


11 


J 


9 




12 


N 


9 


^JL4J. IvC II 


3m J 


Lt 


Q 


Level 


14 


Y 


g 


L L V U a IIO Wl? L 


25 


X 


6 


Used An^iw^r KfiV 


16 


V 


9 


Tnt^ Bnar^l ComDl 6t ion 


17 


M 


0 


chance Mnrl^A tn T)Vii&i]il.c 


18 


P 


6 


Pause 


19 


V 


9 


Triangle 


20 


w 


9 




21 


s 


21 


X X L 


22 


c 


9 


Left Sector Scan 


23 


□ 


9 


Right Sector Scan 


24 


t 


9 


Center Sector Scan 


25 


\ 


9 


Brake Sector Scan 


26 


Q 


12 


Start Problem 


27 


K 


12 


Start Free Fly Problem 


28 


Z 


0 


Stop Problem 


29 


> 


9 


60^20 mile B-Scan Scale 


30 


< 


9 


204-60 mile B-Scan Scale 


31 




9 


Lock One 


32 




9 


Break Lock 


33 


? 


9 


Attempt Lock On » Fall 
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lUilSTRATIONS OF STUDENT DATA STOIHGS 



1-8 (3, A, B, i:, E, P, It 

Define turn sevority and dirtiction- If one ot these 8 Alphas 
is followed directly by 12 digits (not by a D or T) the turn 
command was to turn a specified number of degrees* 

e»g» A So6 Co ^64 

12 3 4 

o 

1- Initial Fighter heading 230 

2, Range 6 miles 

3, Turn specification 40^ 

4, Problem time 64 seconds 

A specifies a port hard turn* 



9 D 

Define turn 360*^ command, preceded by one of Alphas 1-8 

A, B, C, E, F, G, H) for direction and severity specification* 

e*g* GD 007 leo 

1 21 3 

CD turn starboard hard, 360 

1 , Range 7 miles 

2, Initial Fighter Heading 160^ 

3- Problem time 145 seconds 



10 T 

Define a turn to a spocifiod heading* T is always preceded with 
an Alpha 1-8 A, B, C, E, F, C, H) to specify turn severity 
and direction. 

e,g, FT 014 ^^b ^ 

12 3 4 

FT starboard standard turn to specified heading 

o 

1* Initial Fighter heading 170 

2 , Range 14 miles 

3* Turn to specified heading 210^ 

4, Problem time 75 seconds 
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11-13 J, N, 1. 

EasQ^ ttj^liLfti^ 1ovl:1 

*rurn Li; i n progress whOn tlusse Cunc tion^; uaod ; the turn is 
mod iC Led at^ |>or indication. 

^ 

e»g. N 002 350 090 
1 2 3 

N Tighten turn 

1 • Range 2 miles 

2» Current Fighter heading 350*^ 

3. Problem time 90 seconds 



14-15 Y, X 

Y correct answer entered 
X answer key used 

Number of orrors and latency stopped 

e.g. Y 002 612 
1 2 

Y correct answer entered 

1 * Ho errors 2 

2. Latency 12 seconds 



16 U Tote board completion 

e.g. U 02b 003 046 
12 3 

U Tote board completion 

1 . Range 20 miles 

2 . Ho error*i 3 

3. Time co completion of tote board 46 seconds 



17 M indicates transfer from sea tic to d3maniic phase of dynamic 

problem, problem discarded from analysis. 

e.g. m 620 003 046 

Interpreted as U above but problem Is discarded from analysis 
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V Pause 

Use of pause kt*y 

i 2 

1 . Time panfiu tc^rnunatocl 90 seconds 

2. Duratiun oT panao 10 seconds 



V Triangle 

Ceometric display ol intercept triangle 

eg, V 606 0?^ 080 
1 2 3 



1 , Range 6 miles 

2. Duration of triangle use 5 seconds 

3* Problem timC! at end of ujie of triangle 80 seconds 



W Tote Hoard 

Used only in dynamic phast: ol dynamic problem. 
Displays updated version of tote board variables. 

e.g. W 00? 012 065 
1 2 3 



1 . Range 7 miles 

2. Duration of use 12 seconds 

3. Problem time at end of use 65 seconds 



S Fire 

Indicates the termination of a dynamic or free fly problem* 

e.g. S 001 IM 145 00 1 m 001 

1 2 3 4 5 6 7 

1 - Range 1 mile 

2- Hit probability 1007. 

3. Problem time 145 seconds 

4. Range 1 mile 
3. Target aspect 174° 

6. Angle ofC 1*^ 

7. Number of turns 2 



Lult, risiit, t:euter sector scan 



11 - V. ^ 



009 OOy 0^1 B 
12 3 



C j^ntcr 1e£t .sector 



1 . Range 

3 t Problem t ime 



9 miles 
9 miles 
seconds 



26-27 Q, K 

Q StarL of static or dynamic problem 
K Start of L'ree fly problem 

e.g. Q oTg 144 0 I'lOO T 
1 2 3 4 5 

Q Problem start 

1 . Student number 29 

2. Problem number 144 

3. O^dynamic problem 

A 1 here and 0 in 5 would be static 
4* Criterion word 
1100^2 of 4 

TA categories have been consecutively successfully cocnpleted 
5. 6^=:^yaamic 

Speed level 460 kts 



28 



Z indicates end of session 



29-30 < f > B-Scan range change 

< 20-^60 milo range 
>60-?20 mile range 



e*g, 



017 6T7 02^ 
12 3 

> change from 60 to 20 mile range scale 



1, Range 
2« Range 
3* Problem time 



17 miles 
17 miles 
20 seconds 
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31-33 - , ; , ? 

Lock on, break lock, attempt and fail to lock on. 



e.g. = 00^4 015 070 
1 2 3 



= Lock on radar 

1. Range 2 miles 

2. Angle Off 15* 

3. Problem time 70 seconds 
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*i ABSTRACT 



Considerations and procedures for applying Performance-Structure Oriented GAI 
in Naval training are described in terms of a general diagram of the necessary 
elements in a CAI system. 

The fundamental objective is to lead the student to develop his own cognitive 
structures that will serve him for generating the surface structures of tatjks. 
The instructional system must accomplish this by generating and scheduling a 
suitable sequence for each student* 

The developmental steps, from job task-structure analyses to computer programs, 
are described* Techniques for generalizing these procedures fall into two 
categories: higher-level programming languages that would permit more rapid 
production of programs for specific applications, and general -purpose programs 
that accept data modules specific to an application* Both kinds of techniques 
have been used in earlier applications. 

Examples of instructional flowcharts, programmer's flowcharts, and data 
encoding procedures used in the development of the RIO trainer are presented 
to illustrate key points in the developmental steps* 
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